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Background / Context 

Previous research has linked inquiry-based science instruction (i.e., science instruction that 
engages students in doing science rather than just learning about science) with greater gains in 
student learning than text-book based methods (Vanosdall, Klentschy, Hedges & Weisbaum, 
2007; Banilower, 2007; Ferguson 2009; Bredderman, 1983; Shymansky, Hedges, & Woodworth, 
1990). The LASER model, being validated in the current study, has already been the subject of a 
number of case studies (RMC Research Corporation, 2010; Horizon Research, 2010; Vanosdall 
et ah, 2007). However, experimental studies of the type that might establish a causal link 
between program implementation, student science learning, and other valued outcomes have yet 
to be conducted. Only a handful of studies have involved random assignment, and most of these 
have involved random assignment of students in a relatively small number of classrooms (see 
Furtak et al. 2009). With support from the U.S. Department of Education’s Investing in 
Innovation Fund (i3), the current validation study of the EASER Model encompasses 
approximately 60,000 students, 1,900 teachers, and over 140 district administrators and 
principals. 

Purpose 

The purpose of this study was threefold: (1) to determine the impact of the Eeadership and 
Assistance for Science Education Reform (EASER) model on science achievement for students 
in grades 1-8, (2) to examine EASER implementation fidelity across multiple settings, and (3) to 
determine the effectiveness of the LASER model in shifting students’ attitudes toward science. 
This study addressed the following key research questions: 

1. Do students in schools that receive the EASER model over 3 years (i.e.. Phase 1 schools) 
attain higher levels of science achievement than students in schools that do not receive 
the EASER model (i.e.. Phase 2 schools) as measured by: 

a. Partnership for the Assessment of Standards-based Science (PASS)-Basic (structured 
response/multiple-choice questions only)? 

b. PASS -Extended (open-ended & performance tasks in addition to structured response 
questions)? 

c. State assessments? 

2. Do the student achievement results vary by subgroup? 

3. What factors influence the implementation of the EASER model? 

4. What is the impact of participation in the EASER model on student attitudes toward 

science? 

Setting and Participants 

The study sample includes 125 schools across three regions, including rural and urban schools in 
16 districts. (Please insert Table 1 here.) The districts serve primarily Hispanic and White 
populations (44.4% and 25.8%, respectively), with more than two-thirds of students (70.0%) 
identified as “economically disadvantaged”. The study sample is also comprised of 10.2% 
African American students, 2.5% American Indian/ Alaskan Native students, and 1.6 % Asian 
students. A sub sample of approximately 9,000 students who were in fourth or seventh grade 
during the 2012-13 school year are being followed longitudinally over the three years of the 
study, which began in the 201 1-12 school year, and comprise the data analytic sample reported 
on in this proposal. 
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Intervention Program 

Developed by the Smithsonian Science Education Center (SSEC), the Eeadership and Assistance 
for Science Education Reform (EASER) model is intended to improve the quality of classroom 
science instruction through a “systemic” approach that engages participants at every level, from 
classroom teachers up through the highest levels of district, regional, and state leadership. 
EASER employs the STC science curriculum (also developed by the SSEC), a set of kit-based 
instructional units that emphasize inquiry-based instruction, i.e., science instruction that engages 
students in doing science rather than just learning about science. 

Through leadership development programs, participants learn to implement a new vision of 
effective science learning and teaching along with five critical areas of the system or 
infrastructure that need to be in place for improving teaching and student achievement: (a) 
research-based instructional materials using the Science & Technology Concepts (STC) science 
curriculum; (b) differentiated professional development programs that help teachers move from 
novice to competency; (c) aligned and cognitively-demanding assessments; and (d) cost-efficient 
and sustainable systems. The goal of the EASER model, a 5-year intervention system designed 
for students in first through eighth grade, is to provide multi-layered, systemic support to 
improve the quality of science education in participating districts. (Please insert Eigure 1 here). 

Research Design 

This study utilized a mixed-methods approach with school-level randomized matched-pair 
design to examine student science achievement, including qualitative feedback gleaned from 
surveys and case studies. Implementation fidelity, as measured by random observations of 
science lessons, was also a key factor of this evaluation. 

Participating schools in all three regions were matched on the following variables, then randomly 
assigned to treatment and control groups: Percent Proficient State Science Assessments, Percent 
Non-White, Percent Eemale, Percent Economically Disadvantaged, Percent of Students with a 
Disability, Percent of Students who are Eimited English Proficiency, Total number of students. 
Total Number of Suspensions, Attendance Percentage. Treatment schools received immediate 
implementation of the EASER model, while the control group was assigned to delayed 
implementation, and will receive the EASER intervention after the research study has ended. 

Instrumentation 

Student Science Achievement. Two science assessments were used in this study. The primary 
achievement outcome measure is the PASS, selected by CREP and developed by WestEd, which 
is a reliable and valid assessment aligned to inquiry-based science. PASS consists of a 
combination of multiple-choice (PASS-Basic), constructed response (open-ended), and 
performance items (PASS-Extended). Einally, PASS measures student science achievement at 
two grade bands, 3-5 and 6-8, and is thus suitable for both the elementary and middle school 
cohorts in the study. As a secondary measure, we will use state achievement tests, which vary by 
state and have changed recently; therefore, descriptions of those tests will not be included here. 

Implementation Fidelity. STC Unit Eogs were developed by CREP to record STC 
implementation data from science teachers in grades 1-8. The STC Unit Eog asks teachers to 
document the extent to which they were able to teach the unit as designed. The STC Unit Eog 
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provides overall mean ratings from “Not Observed” to “Exeellent” across 20 items on a four- 
point Likert scale asking questions such as, “Did you teach the lessons in the suggested 
sequence?” 

Student Attitudes Toward Science. The PASS also includes 15 items that address students’ 
attitudes towards science, such as how much they like science, if science will be useful to them, 
and how much they discuss science with their family and friends. 

Procedure and Analysis 

The current study was conducted from August 2010 through May 2012. The participating 
districts were identified by the program developers and voluntarily agreed to participate. The 
districts also agreed to implement the LASER model as designed. Treatment schools 
immediately received the LASER model, and baseline data collection began prior to the 
implementation of the model. The control group was assigned to receive the LASER model in 
Summer 2014, after the research study has ended. 

Exploratory analyses of outcomes on the 29 multiple-choice PASS items were conducted to 
determine differences between Phase 1 and Phase 2 students in third grade and sixth grade on 
Spring 2012 PASS-Basic (PASS-B) scaled scores (first post test). Only students who answered at 
least one question were included in the analyses for the respective grade, with each student’s 
scaled score on the Spring 2012 PASS-B used as the outcome measure. Hierarchical or “block 
entry” multiple regressions were conducted taking into account the following covariates: Special 
Education status (lEP), English Language Learner (ELL) status, Eree and Reduced Price Lunch 
(ERL) status, and Eall 2011 (baseline) PASS scaled score. In addition to these regressions, a 
second set of ANCOVA analyses were conducted for all students within grade level, as well as 
for subgroups of these same students (by lEP status, ELL status, ERL status, and Gender) using 
the same covariates. Analyses of Spring 2013 data for both the PASS-Basic and PASS- 
Extended, as well as state achievement test, are currently underway and will be available for 
presenting at the Spring 2014 SREE conference. 

Eor student attitudes towards science and the STC Unit Log implementation measure, descriptive 
statistics have been conducted thus far to examine stakeholder perceptions as well as levels of 
implementation fidelity. To be included in the survey analyses, a student had to have answered 
at least one question in Eall 2011 and one question in Spring 2012. 

Results 

Student Science Achievement. A total of 5,410 third grade students in Phase 1 (n = 3,026) and 
Phase 2 (n = 2,384) schools were included in the achievement analyses. The addition of the 
students’ Phase in the final block of the regression model was statistically significant, and 
favored Phase 1 (Beta =.02, t =1.97, p = .049). However, the effect size (g = 0.04) from the 
ANCOVA demonstrated that the performance edge for Phase 1 was not substantively important 
(i.e. > 0.25) according to What Works Clearinghouse standards (What Works Clearinghouse, 

201 1). All favoring Phase 1, other statistically significant ANCOVA outcomes were observed 
for the 276 lEP treatment students over their 220 controls (E(l, 490) = 4.72, p < .05, g = 0.15), 
the 1,124 treatment students not on Eree/Reduced lunch over their 907 controls (E(l, 2025) = 
9.80, p < .01, g = 0.11), and the 1,531 Male treatment students over their 1,209 counterparts in 
the control group (E(l, 2734) = 4.49, p < .05, g = 0.06). In all three cases; however, the effect 
sizes were not substantively important. 
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For the 2,796 sixth grade students in Phase 1 (n = 1,376) and Phase 2 {n =1,420) schools 
included in the analyses, there were no statistically significant difference in achievement, taking 
into account all of the other variables in the previous blocks {Beta =0.01, t = 0.65, p = .5 14), 
with an accompanying g = 0.02 effect size from the ANCOVA analysis. In addition, there were 
no statistically significant differences observed when subgroup ANCOVA adjusted means were 
analyzed. 

Student Attitudes. In third grade, 2,650 Phase 1 and 1,835 Phase students were included. A vast 
majority of both Phase 1 and Phase 2 students (roughly 90% - 92%) in both Fall 2011 and Spring 
2012 gave affirmative responses, stating “I like science” either “a lot” or “a little”. At both time 
points, while almost half of third grade students across the three regions recognized that science 
would be useful to them when they were older, a much lower percentage (roughly 17% - 24%) 
wanted to be a scientist when they grew up. Furthermore, 3rd grade students across the three 
regions at both time points were more likely to talk to family compared to friends about what 
they did in science class. 

In sixth grade, 1,337 Phase 1 and 1,388 Phase 2 students were included. Overall, a vast majority 
of both Phase 1 and Phase 2 students (roughly 86% - 94%) in both Fall 2011 and Spring 2012 
gave affirmative responses (“I like science” either “a lot” or “a little”). At both time points, 
while almost half of sixth grade students across three regions recognized that science would be 
useful to them when they were older, a much lower percentage (less than 10%) wanted to be a 
scientist when they grew up. Also at both time points, sixth grade students across the three 
regions were more likely to talk to family compared to friends about what they did in science 
class. 

Fidelity of LASER Implementation. Results indicated that 551 STC Unit Logs were completed. 
Across all regions, over 95% of teachers reported teaching the units in the order intended. Over 
85% of teachers answered “completely” or “to a large extent,” when asked if they taught all of 
the lessons/activities in the unit, indicating that most units were also taught in their entirety. 

Only 9.5% of teachers across all three regions reported teaching the unit completely as a teacher 
demonstration, as opposed to allowing the students to work with kits hands-on, as intended by 
the curriculum developers. At least 88% of teachers across all regions reported having all the 
materials needed to teach their units. 

Conclusions and Educational Significance 

Early findings from the current study indicated promising findings at third grade and possibly 
indication that implementing inquiry-based science early in students’ education may produce 
larger growth in science achievement, and may have differential impacts for specific subgroups, 
as evidenced by the positive finding for lEP students. The lack of findings for middle school 
will warrant further exploration if that trend in outcomes continues. The effects on student 
attitudes, however, are less clear, and appear to have less impact to date. It could be that if 
differences in achievement continue, differences in attitudes will follow. The efficacy of the 
EASER program has important implications for both research and practice when working with 
high-poverty schools and districts, who have limited resources and time available for science 
interventions. EASER’s initial success with early learners also demonstrates its potential for 
reducing the development of chronic, long-term deficiencies and academic problems. 
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Appendix B. Tables and Figures 


Table 1. Current Research Study Sites by Condition 


Region 

Schools 

Phase 1 
(Treatment) 

Phase 2 
(Control) 

Houston Independent Sehool District 

50 

26 

24 

New Mexico (Eight districts) 

33 

19 

14 

North Carolina (Seven districts) 

42 

21 

21 

Total 

125 

66 

59 


Figure 1. Theory of Action for the LASER Model 
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